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ABSTRACT 



This first volume of a four-volume evaluation of the 



1988-89 New York City School Community Education Program (also known 
as the Umbrella Program) comprises reports evaluating 10 innovative 
projects that provide reading, mathematics, writing, English as a 
Second Language, and computer literacy instruction c Evaluation 
sources included student preprogram and postprogram test outcomes, 
writing samples, teacher and student questionnaires, and the number 
of acceptances of participants into special high schools. Overall, 
the program was not as successful in meeting its stated objectives as 
in previous years, but the Harlem School-Community Tutorial Project 
and the Mathematics Improvement Program were particularly successful. 
The report on each project contains a brief project overview, 
describes the research methodology, presents the findings, and 
provides recommendations for improvement. The following projects are 
evaluated: (1) Improving Reading Achievement After School; (2) Harlem 
School-Community Tutorial Project; (3) Tutoring — Walk and Talk 
Program; (4) Improving Competency Skills; (5) Mathematics Improvement 
Program; (6) Students-Parents As Partners; (7) Science Enrichment 
Program; (8) Computer Literacy Program; (9) Reinforcing Reading; and 
(10) the Riverdale Preparatory Academy. Statistical data are included 
on 20 tables. Each report also includes examples of evaluation 
instruments . (FMW) 
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SCHOOL COMMUNITY EDUCATION PROGRAM 
IN NEW YORK CITY 
EVALUATION SUMMARY* 



BACKGROUND 

The School Community Education Program (also known as the 
Umbrella Program) , administered by the Division of Curriculum and 
Instruction, provides a variety of educational and training 
experiences to a wide range of participants, including pre- 
schoolers and their parents, and elementary, intermediate, and 
high school students, teachers, and supervisors. The program 
consists of 37 different projects designed to provide innovative 
solurions to local educational and school problems. Ten p*-ojects 
provide basic skills, English as a Second Language, and computer 
literacy instruction; ten focus on social issues and 
environmental studies; seven offer staff development workshops; 
five involve curriculum development, and three are designed for 
prekindergarten ^.hildren. The remaining projects provide 
participants with a variety of educational experiences. 



POPULATION SERVED 

In 1988-89, the program served some 25,000 students, 
primarily elementary school pupils. In addition, the program 
served 1,100 teachers and supervisors and 100 prekindergarten 
children, as well as neighborhood adults in the 32 community 
school districts and selected high schools. Each project 
established different selection criteria for program 
participation . 



PROGRAM OBJECTIVES 

Although program objectives were designed for each specific 
project and therefore varied, most focused on increasing the 
competence of project participants through mastery of specific 
skills and abilities. Most objectives also set quantitative 
criteria to be met by a minimum percentage of participants for 
the program to be considered successful. 



"This summary is based on the final eve.luation report of the 
School Community Education Program in New York City 1988-89, 
prepared by the Office of Research, Evaluation, and Assessment/ 
Instructional Support Evaluation Unit. 



EVALUATION METHODOLOGY 



The evaluation of the program is based on a number of data 
sources: student performance outcom^is on standardized or 
project-developed tests, pupil writing samples, teacher and 
student survey questionnaires, number of acceptances to special 
high schools, and review of five curriculum documents. These 
manuals and lesson plans were sent to different units of the New 
York City Board of Education's Division of Curriculum and 
Instruction for evaluation. Preprogram and postprogram test 
outcomes were compared to determine mean differences and, when 
appropriate, correlated ^-tests and effect sizes were also 
computed to establish statistical significance and educational 
meaningfulness, respectively. The percentage of participants 
meeting quanticative project-set criteria for success was also 
determined. 



FINDINGS 

The 1988-89 evaluation findings indicate that the School 
Community Education Program was not as successful as it had been 
in previous years. Only 15 projects met their stated objectives, 
compared to 19 in 1987-88. In general, those projects providing 
staff development training and curriculum development were the 
most successful. In addition, two projects that provide remedial 
instruction (Harlem School -Community Tutorial Project, and 
Mathematics Improvement Program) were also found to be 
particularly successful. Ths evaluation also showed that 
although some projects met their objectives, these results should 
be treated with caution because of the vagueness of the 
objectives or because the evaluation instiruments could not 
adequately measure project impact v This is a particular problem 
shared by staff development projects that seek to measu^-e teacher 
ability to implement specific teaching skills in the classroom 
without including instruments which measure these skills. 

Four projects were successful in meeting one of their 
objectives, yet unsuccessful in meeting a second objective. 
Sixteen projects did not meet their evaluation objectives, and 
two projects could not be evaluated because test data were 
lacking. As indicated in previous years* e"*aluations, a few of 
these projects need extensive modifications, such as revision of 
testing instruments to avoid ceiling effect, development of 
project activities appropriate for different grade levels, or 
establishment of more stringent participant selection criteria. 
Most of the unsuccessful projects, however, failed to meet their 
objectives because their criteria for success were too stx*ingent 
or because the testing instrument could not adequately measure 
project objectives. In some of these projects, participants 
achieved large mean gains, but the percentage of successful 
participants remained below the percentage established in the 
project-set criterion for success. In some cases, this criterion 
was beyond what could be reasonably expected of program 
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participants. 



RECOMMENDATIONS 

In addition, to the recommendations made for each project, 
the following suggestions are made for the overall improvement of 
the School Community Education Program: 

Closely .monitor those projects that fail to meet their 
stated objectives. 

Assist project staff in making necessary project 
modifications such as the revision of project activities, 
revision or replacement of testing instruments, 
establishment of adequate selection criteria of 
participants, or amendments in project objectives. 
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* INTRODUCTION 

In 1988-89, the New York City Public Schools received 
$2,375,000 in funding from the New York State Legislature to 
operate the School Community Education program (also known as the 
Umbrella program) . It consisted of 37 different projects 
designed to provide innovative solutions to local educational and 
school programs. 

The program provided services to about 25,000 participants 
in 32 community school districts and selected high schools. 
While most of these participants were elementary school students, 
the program also served some 1,000 intermediate and high school 
students, 100 preschool children, and 1,100 teachers and 
supervisors. Some projects also included parenting components 
and/or sought to involve the parents of participating students in 
project activities. 

Evaluation reports are presented in four volumes. Volume I 
contains evaluations of ten projects that provided reading, 
mathematics, writing, English as a Second Language, and computer 
literacy instruction. Volume II includes evaluations of nine 
projects on social, ethnic, and environmental studies* Four of 
these projects also provided staff development workshops. Volume 
III contains evaluations of seven staff development and five 
curriculum development projects. The remaining six projects, 
presented in Volume IV, offered a variety of educational 
experiences to participants. Three of these projects werr. 
designed for prekindergarteu children, and the other three 
projects were designed to teach students health maintenance 
concepts, to improve their acceptance rate to special high 
schools, and foster career awareness among students. 

Each report contains a brief project overview, describes the 
evaluation methodology, presents the findings, and provides 
recommendations for improvement. The reports are listed in order 
of budgeted function number in the Table of Contents. 



TABLE OF CONTENTS 



VOLUME I 



Project Title Project Number 

1 Improving Reading Achievement After 

School 93402 

2 Harlem School-Community Tutorial 

Project 93406 

3 Tutoring: Walk and Talk Program 93409 

4 Improving Competency Skills 93410 
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6 Students-Parents as Partners 93422 

7 Science Enrichment Program 93425 
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IMPROVING READING ACHIEVEMENT AFTER SCHOOL, 1988-89 

School-Coininunity Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Michael D. Carlin 

James Campbell 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Improving Reading Achievement After School project is 
designed to provide basic skills training to first- through 
sixth-grade students in Community School District (C.S.D.) 5. By 
exposing participants to diverse communication arts activities, 
the project seeks to enrich daily classroom experiences and 
improve students reading achievement and self-esteem. The 
program objective was for participants to improve their reading 
ability as measured by annual citywide tests of reading 
achievement. More specifically, the objective anticipated that 
participating students would achieve a statistically significant 
mean gain. 

In 1988-89, the program served 100 elementary school 
students from P.S. 46 and P.S. 123 who had behavior and 
achievement problems and were referred for participation by 
school staff, social workers, or counselors. Activities took 
place three hours a day, five days a week. Teachers, educational 
assistants and student aides, working under the supervision of 
project coordinators, provided basic skills activities to 

ERIC "17 
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individual and/or small groups of students in order to encourage 
them to read; write ; and express themselves orally. Project 
activities also included "hands-on" experience in the arts. 
Pupils were able to choose one c»rt area among those offered at 
the center; music ; dance ; drama ; and visual arts. Other 
activities included visits to museums, libraries, and art 
performances. The New York State Legislature provided $29 
thousand in funding to purchase educational supplies and cover 
admission to museums and other cultural activities. 

EVALUATION METHODOI>OGY 

The impact of the program on student performance was 
determined by analyzing the change in students' reading 
achievement scores from 1988 to 1989. Two different tests were 
administered to participants. The Metropolitan Achievement Test 
(M.A.T.) was given to students in grades one and two in April 
1988 and, again, to second graders in April 1989. The Degrees of 
Reading Power (D.R.P.) Test was administered to students in 
grades three through six in the spring of 1988 and 1989. Since 
those students who were in grade three in April 1987 took two 
different tests (the M.A.T. in 1988 and the D.R.P. in 1989), 
D.R.P. posttest scores were converted to comparable test scores 
on the M.A.T. Scores for all grades were then converted to 
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normal curve equivalent (N.C.E,)* scores which express student 
performance relative to a national norm. Mean N.C.E. gains were 
interpreted as a measure of project impact on student 
achievement. 

Correlated t-tests were computed to establish if achievement 
gains were statistically significant. Effect size (E.S.)** which 
indicates the educational meaningful ness of the mean gain or loss 
for each comparison was also calculated. 
FINDINGS 

Complete test scores were submitted for 49 students from 
P.S. 46. Overall, mean pretest score was 50.5 N.C.E. points, 
mean posttest score was 46.4 N.C.E. s, for a mean loss cf 4.1 
N.C.E. score points (see Table 1). There was wide variation in 
student performance across grade levels and within the same 
grade. Individuals gains ranged from -47 to 65 N.C.E. points. 



N.C.E. scores are similar to percentile ranks, but unlike 
percentile ranks, are based on an equal-interval scale. Nomnal 
curve equivalent scores are based on a scale ranging from 1 to 99 
with a mean of 50 and standard deviation of approximately 21. 
Because N.C.E. scores are equally spaced apart, arithmetic and 
statistical calculations such' as averaging are meaningful; in^ 
addition, comparisons of N.C.E. scores may be made across 
different achievement tests. 

The effect size, developed by Jacob Cohen, is the ratio of the 
mean gain to the standard deviation of the gain. This ratio 
provides an index of ivitprovement in standard deviation units 
irrespective of the size of the sample. According to Cohen, 0.2 
is a small E.S., 0.5 is a moderate E.S., and 0.8 is considered to 
be a large E.S. Only effect sizes of 0.8 and above are 
considered to be educationally meaningful, reflecting the 
importance of the gains to the students' educational development. 
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TABLE 1 

Students' Mean N.C.E. Scores by Grade 
Metropolitan Achievement Test And Degrees of Reading Power Test 
Improving Reading Achievement After School, 1988-89 







pretest 


Posttest 


Difference 




Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


E.S. 


3 


11 


54.2 


16.1 


38.1 


22.6 


-16.1 


10.5 


-1.5 


4 


13 


48.5 


17.3 


45.2 


14 .9 


-3.3 


15.6 


-0.2 


5 


11 


44.6 


19.5 


52.6 


15.2 


8.0 


25.2 


0.3 


6 


13 


56.2 


7 . 3 


59.4 


7.5 


3.2 


7.4 


0.4 


Total 


48 


50.5 


16.4 


46.4 


16.9 


-4.1 


18.1 


-0.2 



Overall mean gain was -4.1 N.C.E. points. This mean gain 
was not statistically significant. 

Student performance varied widely between grades. 
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Students in grade 5 achieved the largest mean gain of 8 N.C.E.'s. 
Seventy-three percent of fifth-grade students achieved gains on 
posttest . 

CONCLUSIONS AND RECOMMEHDATIONS 

In 1988-89, the Improving Reading Achievement After School 
project was not successful in meeting its objective. Overall 
mean gain was -4.1 N.C.E. points. There was wide variation in 
student performance across grade levels and within the same 
grade. Students in grade five achieved the largest mean gain 
(8.0 N.C.E. points). Individual student gains ranged from -47 to 
65 N.C.E. points. This variation in student achievement 
indicates the uneven effectiveness of the program in 1988-89. 

A comparison of mean pretest scores by grade indicates that 
different criteria were used to select participants. In general, 
students from grades three, four, and six were performing well 
before the program began, and their problem may have been their 
behavior or attitude towards school rather than on their need for 
remedial instruction. In contrast, it seems that for grade five 
priority was given to students who showed the greatest 
educational need for basic skills remediation. Evaluation 
findings indicate that using two different sets of criteria to 
select participants precludes the ability of a project to 
simultaneously remediate the needs of two distinct categories of 
students. In order to respond to student's needs more 
effectively, the project coordinators should discuss the type of 
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students the program aims to serve and modify project activities 
according to these criteria. 
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HARLEM SCHOOL-COMMUNITY TUTORIAL PROJECT, 1988-89 

School-Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Edythe B. Edwards 

Prepared by: 
Office of Research/ Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Harlem School -Community Tutorial Project provided 
intensive basic skills instruction in reading and mathematics to 
students in Community School District (C.S.D.) 5. Pupils in 
grades two through twelve ; in need of basic skills remediation, 
were selected on the basis of diagnostic tests and 
recommendations made by school personnel and parents. Individual 
and small-group tutorials were held at two neighborhood community 
centers from 2:00 to 6:30 p.m. Students were expected to receive 
at least three hours of remedial instruction per week. 

In 1988-89/ some 224 students completed the program, 139 in 
reading and 85 in mathematics. The program objective was for 
participants to achieve a statistically significant mean gain in 
their performance in reading or mathematics, as measured by 
citywide standardized tests. Project staff consisted of a 
program coordinator, ten teachers and four educational 
assistants. The New York State Legislature provided $29 thousand 
in funding for the project. 
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EVALUATION METHODOLOGY 

Project impact was assessed by analyses of student 
performance on the reading or mathematics subtests of the 
Metropolitan Achievement Test (M.A.T.)* Students took the 
appropriate grade-level form of the M.A.T, as a pretest upon 
entering the program, students entered at different periods in 
the school year. All students were posttested in June when 
program activities concluded. In order to compare test scores of 
students who began project activities at different times, grade 
equivalent scores were converted to scale scores, since there are 
not mid-year norms for the M.A.T. Correlated t-tests were 
computed to establish if achievement gains were statistically 
significant. Effect size (E.S.)/* which indicates the 
educational meaningfulness of the mean gain or loss for each 
comparison, was also calculated. 
FINDINGS 

Students in most grades achieved statistically significant 
mean gains on both the reading and mathematics subtests of the 
M.A.T. Table 1 reports student performance on the reading 



*The effect size, developed by Jacob Cohen, is the ratio of the 
mean gain to the standard deviation of the gain. This ratio 
provides an index of improvement in standard deviation units 
irrespective of the size of the sample. According to Cohen, 0.2 
is mall E.S.; 0.5 is a moderate E.S., and 0.8 is considered 
t( ^e a large E.S. Only effect sizes of 0.8 and above are 
considered to be educationally meani»^gful, reflecting the 
importance of the gains to the students' educational development. 

2 
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subtest for all grades. Overall, mean pretest score was 682.3 
scale points, mean posttest was 720.5 points, for a mean gain of 
38*2 scale points* This mean gain was found to be statistically 
significant and educationally meaningful. Mean gains varied by 
grade and ranged from 15 to 50.1 scale points. Mean gains for 
all grades, except grades 9 and 11, were statistically 
significant. Students in grade nine Achieved a mean gain of 47.5 
scale score points although this mean gain was found to be only 
marginally significant (p=.06) due to the extremely small sample 
size. 

Table 2 presents students* mean scores on the Mathematics 
test, by grade. Overall mean pretest score was 654.9 scale score 
points, mean posttest score was 693.9 points, for a mean gain of 
39 scale points. This mean gain was found to be statistically 
significant and educationally meaningful. All grades, with the 
exception of grade 11, achieved statistically significant mean 
gains. 

CONCLUSIONS AND RECOMMENDATIONS 

The evaluation findings indicate that the Harlem School- 
Community Tutorial Project was a successful program in 1988-89. 
Students improved their performance on both the reading and 
mathematics subtests of the M.A.T. In general, pupils achieved 
large gains. These gains were educationally meaningful for all 
grades and statistically significant for most grades in both 
subtests. 

3 
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TABLE 1 

Students' Mean Scaled Scores on the Reading Subtest 
of the Metropolitan Achievement Test, by Grade 
Harlem School-Community Tutorial Project, 1988-89 







Pretest 


Posttest 


Difference 




Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


E. S . 


3 


15 


603 .2 


42.6 


653.3 


32. 1 


50. 1* 


19.6 


2.6 ■ 


4 


21 


621.6 


56.2 


669.3 


42.7 


47. 7* 


28.4 


1.7 


5 


34 


675.0 


30.5 


705.4 


25.5 


30.4* 


14.0 


2.2 


6 


18 


690.8 


39.6 


728.1 


40.8 


37.3* 


15. 3 


2.4 


7 


25 


720.2 


59.9 


760.2 


59.0 


40.0* 


25.2 


1.6 


8 


12 


712 .4 


88.2 


745.1 


85.8 


32.7' 


15.4 


2.1 


9 


2 


757 .0 


17.0 


804.5 


23.3 


47, .5 


6.4 


7.4 


10 


9 


744 .5 


67.5 


780.2 


43.1 


35 .7* 


31. 1 


1. 1 


11 


3 


860.7 


77.4 


875.7 


64.5 


15.0 


13.1 


1.1 


TOTAL 


139 


682 .3 


72.7 


720.5 


65.2 


38.2* 


21.9 


1.7 



^These gains were significant at p<.05. 

Students in all grades, except grades 9 and 11, 
achieved statistically significant mean gains. These 
gains were also educationally meaningful. 
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TABLE 2 

Students' Mean Scaled Scores on the Mathematics Subtest 
of the Metropolitan Achievement Test, by Grade 
Harlem School-Community Tutorial Project, 1988-89 



Pretest Post test 

















o . u . 




3 


4 


561 . 7 


29 . 0 


602 . 0 


19 . 4 


• 

40.3* 


13.1 


3 . 1 


4 


12 


565.3 


58.0 


615.2 


35. 3 


49.9* 


28.9 


1.7 


5 


15 


633.7 


33.5 


674.5 


16.1 


4C.8* 


24.8 


1.6 


6 


11 


659.1 


44.2 


701.0 


28. 3 


41.9* 


21.7 


1.9 


7 


16 


671.4 


40.9 


711.3 


31.4 


39.9* 


22.6 


1.8 


8 


10 


665.4 


76.1 


701.9 


57.1 


36.5* 


23.3 


1.6 


9 


8 


707.6 


38.7 


734.4 


23.6 


26.8* 


22.9 


1.2 


10 


6 


719.2 


66. 3 


754.2 


62.2 


35.0* 


19.7 


1.8 


11 


3 


836.3 


101.2 


855.7 


82.4 


19.4 


21.2 


0.9 


TOTAL 


85 


654.9 


76.7 


693.9 


63.7 


39.0* 


23.5 


1.7 



'These gains were significant at p<.05. 

Students in all grades except grade eleven, achieved 
statistically significant gains. These gains were also 
educationally meaningful. 
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Analyzed by grade, however, mean gains ranged from 15 to 50 
scale points on the reading subtest, and from 19 to 50 scale 
points on the mathematics subtest. The range of individual 
student performance was even larger, from -12 to 115 scale score 
points on the reading subtest and from 0 to 114 points on the 
mathematics subtest. The range of student performance indicates 
that the project had a varied impact on in'dividual students. In 
the future, project staff may wish to intensify individualized 
instruction, especially for those students in greatest need. 
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TUTORING: WALK AND TALK PROGRAM, 1988-89 ff 

School -Community Education Program 
Program Administrator: Morris Speiser 
Project Coordinator: Eileen Eng 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Tutoving: Walk and Talk Program provides individualized 
and small-group afterschool instruction in English as a Second 
Language to students in three elementary schools in Community 
School District (C.S.D.) 2. Approximately 200 pupils in grades 
one through six participated in the afterschool program during 
1988-89. The students were recent immigrants mainly from Cnina 
and Southeast Asia. "Walk and Talk" field trips are used to 
acquaint participants with New York City and encourage them to 
spea^c English. 

Participants were selected on the basis of teacher 
recommendations, scores on the Language Assessment Battery (IjAB) 
Test, and other measures of language proficiency. Project 
activities included an orientation to American culture, 
individual and group work in reading language arts, assistance 
with homework, and "Walk and Talk" trips. Contact time ranged 
from 80 to 160 hours, depending on students' needs. 

The program objective was for participants to improve their 
language skills by achieving a statistically significant mean 
gain as measured by pre- and posttest on the Language Assessment 
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Battery (LAB), a standardized test of language achievement. 
Project staff consisted of one coordinator, licensed teachers, 
educational assistants, student aides, and a family worker. The 
program received $22 thousand in funding from the New York State 
Legislature. 



EV ALUATION METHODOLOGY 

Project impact was assessed by analyses of students' scores 
on the Language Assessment Battery Test. This test was 
administered to all participants on a pretest and posttest basis 
at the beginning and end of the program. Raw scores were 
converted to normal curve equivalent (N.C.EO* scores which 
express student performance relative to national English- 
proficient norms. Mean N.C.E. scores were compared and 
correlated t-tests were computed to establish if achievement 
differences were statistically significant. The effect size 
(E.S.)", which indicates the educational meaningfulness of the 



*N.C.E. scores are similar to percentile ranks, but unlike 
percentile ranks, are based on an equal-interval scale. Normal 
curve equivalent scores are based on a scale ranging from 1 to 99 
with a mean of 50 and a standard deviation of approximately 21. 
Because N.C.E. scores are equally spaced apart, arithmetic and 
statistical calculations such as averaging are meaningful; in 
addition, comparisons of N.C.E. scores may be made across 
different achievement tests. 

*The effect size, developed by Jacob Cohen, is the ratio of the 
mean gain to the standard deviation of the gain. This ratio 
provides an index of improvement in standard deviation units 
irrespective of the size of the sample. According to Cohen, 0.2 
is a small E.S., 0.5 is a moderate E.S., and 0*8 is considered to 
be a large E.S. Only effect sizes of 0.8 and above are 
considered to be educationally meaningful, reflecting the 
importance of the gains to the students' educational development. 
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mean gain or loss for each comparison, was also calculated. 

FINDINGS 

Complete test scores were available for 177 pupils. Table 1 
reports the performance of students on the Language Assessment 
Battery Test. Overall, mean pretest score was 10.5 N.C.E. 
points, mean posttest score was 22.9 N.C.E. points, for mean gain 
of 12.4 N.C.E. score points. Mean gain scores by grade ranged 
from 8.6 N.C.E. 's to 28.9 N.C.E. 's. Mean gains for all grades 
were statistically significant and educationally meaningful. 
Individual student performance varied widely between and within 
grades as indicated by the large standard deviations of mean 
posttest scores and mean gains. Gains achieved by participants 
ranged from a negative difference between posttest and pretest 
scores of minus seven N.C.E.s to a gein of 65 N.C.E.s. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Tutoring: Walk and Talk Program was 
successful in meeting its objective. All grades achieved mean 
gains and these mean gains were statistically significant and 
educationally meaningful. Mean gain ranged from 8.6 N.C.E. 's for 
grade five to 28.9 N.C.E. 's for grade one. This variation in 
performance is particularly notorious in the gains students made; 
these gains ranged from a negative gain or no gain to a large 
gain o^C 65 N.C.E.s. 

On the basis of these evaluation findings, it would seem 
that students would benefit even more from their participation in 

3 
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TABLE 1 

Students' Mean N.C.E. Scores on the Language 
Assessment Battery Test, by Grade 
Tutoring: Walk and Talk Program, 1988-89 



Pretest Posttest Difference * 

Grade N Mean S.D. Mean S.D* Mean S.D. E^S, 



1 


13 


9.7 


6.8 


38.6 


16.9 


28.9* 


13.0 


2.2 


2 


13 


6.3 


9.7 


20.8 


5.8 


22.5' 


18.4 


1.2 


3 


36 


15.4 


9.6 


28,1 


9.0 


12.7' 


7.9 


1.6 


4 


45 


10.9 


7.7 


20. G 


10.8 


9.7' 


9.8 


1.0 


5 


41 


6.9 


4.3 


15.5 


9.4 


8.6' 


8.0 


1.1 


6 


29 


11.2 


10.8 


20.8 


13.4 


9.6' 


8.7 


1.1 


TOTAL 


177 


10.5 


8.6 


22.9 


13.5 


12.4' 


11.5 


1.1 


'These 


mean 


gains 


were significant 


at p<. 


05. 







* Mean gains for all grades were statistically 
significant, and educationally meaningful. 

Overall; mean gain was 12.4 N.C.E. scores. 

* First-grade pupils achieved Tch^ largest mean gain. 
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the program if they were grouped in sessions according to their 
pretest scores rather than by grade. Alternatively, project 
staff may wish to provide more individualized instruc^.ion to 
those pupils who seem to make little progress. 



5 
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IMPROVING COMPETENCY SKILLS, 1988-89 

School -Comxnunity Education Program 
Program Administrator: M* Morris Speiser 
Project Coordinator: Judith D. Murphy 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit' 
New York City Public Schools 

PROJECT DESCRIPTION 

The Improving Competency Skills program is designed to 
provide basic reading and mathematics skills instruction to in- 
and out-of -school youth and adults* The goal of the project is 
to help participants further their education and improve their 
competency at seeking better educational and employmert 
opportunities* Using an individualized, diagnostic-prescriptive 
approach, the project also seeks to prepare participants for 
passing the General Education Development Test, Civil Service 
Exams, and similar entrance examinations. 

In 1988-89, about 40 youths and adults participated in the 
program. Adults were chosen from those who needed to develop 
their skills in reading and/or mathematics* High school 
students in grades nine to twelve, were recommended for 
participation in the program by school teachers ^ counselors and 
parents* Those applicants showing the greatest educational need 
were given placement priority* The project objective was for 80 
percent of participants to achieve an increase of at least 20 
percent in reading or mathematical skills as measured by the 
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General Educational Development (G.E.D.) Practice Test. 

Classes for adult participants were held at the Bronx Center 
f or ^Youth two nights a week for two hours. High school students 
attended two-hour sessions at Truman High School twice a week 
after regular school hours. To facilitate individualized 
instruction, each session had less than 20 participants. The 
program curriculum consisted of mathematics concepts, arithmetic 
operations, and computer-based instructional activities. It also 
focused on reading comprehension, vocabulary development, and 
word usage skills. Classroom materials included education kits, 
G.E.D. texts, duplicating materials, and minicomputers. Project 
staff consisted of three teachers and one supervisor. The New 
York State Legislature contributed $6,600 in funding. 
E '/ALU ATI ON METHODOLOGY 

The evaluation of the project was based on analyses of 
participants' scores on the mathematics and reading subtests of 
the G.E.D. Practice Test. The mathematics subtest consists of 25 
problems with multiple-choice responses and the reading subtest 
consists of 20 multiple-choice items based on selections from 
diverse reading materials. Both tests were administered on a 
pre- and posttest basis at the beginning and end of the project. 
Mean gains for participants with complete test information were 
computed and compared. In addition, the number of students 
meeting the project criterion for success was also determined* 



2 



29 



93410 

FINDINGS 

Complete test scores were submitted for 4 0 participants for 
the mathematics subtest and for 20 participants for the reading 
subtest. Table 1 shows test outcomes on the mathematics subtest, 
overall, mean pretest raw score was 11.2 points (44c 8 percent 
correct responses), and mean posttest raw score was 15.8 points 
(63.2 percent correct), for a mean gain of 4.6 raw score points 
or 18.4 percent. No reading scores were provided for 
participants at Truman High School. Only those participants from 
the Bronx Center for Youth were administered the reading subtest 
of the G.E.D. practice test. Mean pretest raw score was 10.0 
points (50 percent correct responses) , and mean posttest raw 
score was 13.4 points (67 percent correct), for a mean gain of 
3.4 points or 17 percentage points. 

Table 2 presents the percentage of students meeting the 
project-set criterion for success. Overall, forty-seven percent 
of all students taking the mathematics subtest met or surpassed 
the project-set criterion for success. Truman High School had 
the largest percentage of successful students (75 percent)* On 
the reading subtest, 40 percent of the students at the Bronx 
Center for Youth met or surpassed the project-set criterion. No 
test scores for the reading subtest were submitted from Truman 
High School. 
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TABLE 1 

Participants' Mean Raw Scores* on the Mathematics Subtest 
of the General Educational Development 
Practice Test, by Site 
Improving Competency Skills, 1988-89 



Site 



Pretest Mean 
Raw Percent 
N Score Correct 



Posttest Mean 

Raw Percent 
Score Correct 



Mean Gain 

Raw Percent 
Score Correct 



Bronx Center 
for Youth 20 



11.2 44.8% 



14.1 56.4% 



2.9 



11.6% 



Truman High 
School 20 



TOTAL 



40 



11.2 44.8 
11.2 44.8 



17.5 70.0 6.4 25.6 

15.8 63.2 4.6 18.4 



High School 



Perfect raw score = 25. 

'^Adults attended classes at the Bronx Center for Youth, 
students attended classes at Truman High School. 

Overall, participants achieved a mean gain of 4.6 
raw score points or 18.4 percent increase. 



ERLC 



31 



TABLE 2 



Percentage of Participants Meeting Project-Set Criterion* 
on the General Educational Development Practice Test 
Improving Competency Skills, 1987-88 











Meetina Criterion 




Site 


Subtest 


N 


N 


% 


Bronx center 


Reading 


20 


• 

8 


40.0 


for Youth 












Mathematics 


20 


4 


20.0 


Truman High 










School 


Mathematics 


20 


15 


75.0 



^Eighty percent of participants will achieve an increase of at 
least 20 percent in reading or mathematics skills. 

Forty-seven percent of all students taking the 
Mathematics subtest met or surpassed the project-set 
criterion for success. 
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CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Improving Competency Skills project did not 
meet its objective of 80 percent of student participants 
increasing their performance on either the mathematics subtest or 
reading subtest by at least 20 percent at posttest* Only 40 
percent of participants on the reading subtest and 47 percent of 
participants on the mathematics subtest met ^che project-set 
criterion. On the Mathematics subtest, seventy-five percent of 
participants at Truman High school met or surpassed the project- 
set criterion. The greater success of the program at Truman High 
School is not surprising considering that these participants were 
also attending regular classroom instruction. 

Mean gains were relatively low for adult participants at the 
Bronx Center for Youth. This finding seems to indicate that 
project activities did not provide sufficient remedial 
instruction to participants or that the test used does not 
reflect the instruction provided. In the future, project staff 
should give more emphasis to individualized instruction. The 
number of participants per session should also be reduced. In 
addition, program staff should correlate the skills present in 
the G.E.O. test with the skills taught in the curriculum. 
Finally, test scores should be submitted for both subtests. 
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MATHEMATICS IMPROVEMENT PROGRAM; 1988-89 

School -Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Akhtar Khan 

Prepared by: 
Office of Research/ Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The purpose of the Mathematics Improvement program is to 
provide support services to teachers in elementary and 
intermediate schools in Community School Districts (C.S.D.s) 14 
and 32. These services include diagnostic information and 
prescriptive techniqvies designed to improve student growth in 
mathematics. In 1988-89; the program served 120 teachers and 
3; 480 students in grades two through eight from the 12 elementary 
and five intermediate schools in C.S.D. 32. In addition, six 
teachers from C.S.D. 14 were selected by school principals to 
participate in a lesson plan writing activity. 

Both district and Board of Education staff trained 
participating teachers. Staff development training workshops 
focused on the development of effective prescriptive techniques. 
Computers were used to provide individual information about 
student needs, progress, and mastery of specific mathematics 
skills. The project objective was for the overall percentage of 
participating pupils scoring at or above grade level on the New 
York City standardized Mathematics test to increase by at least 5 
percent as compared to 1987-88. 
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The New York State Legislature contributed ^62 thousand to 
support project activities • The bulk of these funds were used in 
C.S.D. 32 for instructional supplies, data processing equipment, 
and equipment repairs. The remaining funds were used by teachers 
in C.S.D. 14 to develop laathematics curriculum materials. The 
project also shared the resources of the Comprehensive 
Instructional Management System (CIMS) program. 

EVALUATION METHODOLOGY 

Evaluation activities focused on the analysis of C.S.D. 32 
student performance on the Metropolitan Achievement Test (MAT) . 
Students took the test in the spring of 1987 and the spring of 
1988. Students' raw scores were converted to normal curve 
equivalent (N.C.E.)* scores which express student performance 
relative to a national norm. Mean N.C.E. gains were interpreted 
as a measure of project impact on student achievement. 

It was not possible to evaluate the program utilizing the 
current objective of a five percent increase of students scoring 
at grade level as compared with 1987-88 student performance 
because the 1987-88 data reported to the Office of Research 



*N.C.E. scores are similar to percentile ranks, but unlike 
percentile ranks, are based on an equal-interval scale. Normal 
curve equivalent scores are based on a scale ranging from 1 to 99 
with a mean of 50 and a standard deviation of approximately 21. 
Because N.C.E. scores are equally spaced apart, arithmetic and 
statistical calculations such as averaging are meaningful; in 
addition, comparisons of N.C.E. scores may be made across 
different achievement tests. 
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Evaluation and Assessment was not in the form of Grade 
equivalents and cannot be obtained. As a result, the project was 
evaluated according to the objective utilized in 1987-88 which 
states that "50 percent of program participants will achieve a 
gain of at least 5 N.C.E.'s at posttest." 

The completed documents produced by the teachers in C«S.D. 
14 were reviewed by the district's Mathematics Coordinator 
according to an Evaluation Checklist (see Appendix A). The 
Evaluation Checklist is designed to establish the effectiveness 
of the documents and determine if they meet the requirements of 
the New York State Regents, the State Education Department, and 
the New York City Board of Education* 
FINDINGS 

Complete test scores on the IIAT were reported for 1,208 
students from 12 elementary and five intermediate schools in 
C»S»D. 32 ♦ Table 1 presents students* mean N»C.E. gains by 
grade* Overall mean gain was 8.4 N.C.E* score points. Grade two 
students achieved the largest mean gain of 24 N.C.E* score 
points* 

Two documents were produced by C.S»D» 14; one for use in 
grade two and another for use in grade five. Both documents were 
rated positively by the reviewers in all categories on the 
document review checklist. The reviewers stated that many of the 
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Table 1. 

Students Mean N.C.E. Gains on the MAT 
by Grade 

Matliematics Improvement Program, 1988-89 



Grade 


N 


Mean 
N.C.E. Gain 


2 


254 


24.0 


3 


147 


9.2 


4 


180 


10.9 


5 


176 


4.8 


6 


150 


1.3 


7 


137 


-2.4 


8 


164 


0.3 


Total 


1208 


8.4 


Overall mean gain was 


8.4 N.C.E. 


score points. 



* Second grade students had the largest mean gain. 




Table 2. 
Percentage of Students Meeting 
Project-Set Criterion*, by Grade 
Mathematics Improvement Program, 1988-89 



Grade 


N 


Meetina Criterion 
N % 


2 


254 


• 214 


84.2% 


3 


147 


88 


59.9 


4 


180 


120 


66.7 


5 


176 


87 


49.4 


6 


150 


56 


37.3 


7 


137 


33 


24. 1 


8 


164 


54 


32.9 


Total 


1208 


652 


54.0 


"students will 


achieve a five N.C.E. 


gain from pretest 


to 



posttest . 

Overall, 54 percent of participants met or surpassed 
the project-set criterion* 

Second grade students had the largest percentage of 
students who met tha project-set criterion* 
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requirements of the New York State Regents, the State Education 
Department, and the New York City Board of Education were met. 
For the grade two document, the reviewer stared that although the 
manual did integrate reasoning/thinking skills activities this 
section could be expanded. No other comment was provided. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Mathematics Improvement Program was 
successful in meeting its objectives of producing instructional 
materials to assist teachers in diagnostic prescriptive 
instruction. Two manuals were produced, one for grade two and 
another for grade five. Both manuals were rated positively on 
all areas of the Document Review Checklist. 

It was not possible to evaluate the program's second 
objective of a five percent increase of students scoring at grade 
level as compared with 1987-88 student performance because the 
1987-88 data reported to the Office of Research Evaluation and 
Assessment was not in the form of Grade equivalents and it was 
not possible to attain that information. As a result the 
project was evaluated by means of the objective utilized in 1987- 
88 which states that "50 percent of program participants will 
achieve a gain of at least 5 N.C.E.'s at posttest." Evaluation 
in accordance with this revised objective shows that this 
component of the program was successful. Over 54 percent of 
participating students achieved gains of 5 or more N.C.E.s at 
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posttest. There was a wide variation of scores between grades. 
In general, performance declined as the grade level rose. This 
finding parallels the findings of previous years and suggests 
that the program should concentrate more on grades six, seven, 
and eight since students in these grades appear to have the 
greatest need of remedial instruction. 

It is recommended that the objective be revised. The 
current objective of a five percent increase in students scoring 
at grade level compared to the previous year's evaluation is not 
an accurate measure of student achievement in the current year. 
It ignores the individual student's pretest level. If you have a 
great number of students performing at grade level on pretest, 
evaluating performance using the current objective would yield no 
information as to gams these students may have achieved. It is 
also possible to have students who score below grade level on 
posttest but who achieved large gaini*. Again, the evaluation 
using the current objective would not recognize these large 
gains. Therefore, it is recommended that the objective be 
revised to state "students will achieve a statistically 
significant mean gain at posttest." 
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Citywide Ortorella Program ^^^^^93^1^ 
Evaluation Report for CXarriculum Projects Manuals 5#-5-xx 
and Other Documents* (1988>89) 



umbrella Program Name: Date: 



Name of (^e^o Conpleting the Review: 



Ti tie : ^ 

Introduction 

Ihe State Fducation Department requires that all Utbrella Programs 
be evaluateed. in ordtr to help us meet this requirement, we are asking 
that you examine this document, and evaluate it using this form. Thank you 
for your cooperation. 



1. The manual follows the Y N NA 

. w York State syllabus and 
cne New York City curriCvilum. 

Explain: 



2. Ihe manual includes information Y N NA 

and requirements indicated by the 
Regents Action Plan. 

Expla in : ^ 



3. Ihe manual integrates reasoning/ Y N tiA 

thinking skills activities. 

Expla in : 



4. Ihe manual contains lesson plans Y N NA 

that present suitable strategies 
for achieving reason le goals. 

Explain: 
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5, Ttie manual contains objectives and 
concepts that are clearly defined. 



Explain: 



6. The nanual contains classro<xn 
activities and materials that 
are relevant and consistent with 
the stated objectives and teaching 
strategies. 

Expla in : . , 



7. Ihe manual contains criterion 
referenced tests that include 
higher-level thinking questions. 

Explain: , 



8. Ihe manual contains technical Y 
and non-technical language that is 
consistent with the highest standards 
of the Office of Professional De>;elopiTent 
and Leadersnip Training. 

Explain: - 

97' Ihe manual coula be circulated citywide. Y 
Explain: 



10. The manual meets the goals specified in 
the objective of the original prop* ;al. 

Explain: 



#5001-48-93422 

STUDENTS-PARENTS AS PARTNERS, 1988-89 

School -Community Education Program 
Program Administrator; M. Morris Speiser 
Project Coordinator: Melineze Lenhardt 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

\ 

PROJECT DESCRIPTION 

The Students-Parents as Partners program provides after 
school instruction in reading and mathematics to pupils in grades 
three through six at P.S. 54 in Community School District 
(C.S.DO 13. The purpose of the program is twofold. First, it 
seeks to provide children with an additional opportunity to 
improve their educational skills. Second, it is designed to work 
with parents who volunteer as English tutors to help limited 
English proficient (L.E.P.) children function better in the 
society in which they live. Students receive individual i^ied and 
small-group instruction based on a diagnostic-prescriptive 
approach. A second program component trains parent volunteers as 
English as a Second Language (E.S^LO tutors. 

In 1988-89, some 37 students participated in the program. 
Program students were selected on the basis of their need for 
remedial instruction as determined by their schools' records, as 
well as by r3ferrals from school principals, guidance counselors, 
and teachers. Parents were recruited through recommendations 
from the community relations teacher. Five parents were trained 
as tutors in E.S.L. classes. Students attended two-hour after 
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school sessions twice a week. Project staff consisted of a 
teacher, a parent program assistant, educational assistants, and 
a teacher-trainer. The New York State Legislature provided $6 
thousand to fund the project. 

The project objective for 1988-89 was for 75 percent of 
participants to demonstrate a 30 percent improvement in 
mathematics and language arts, as measured by their performance 
on project-developed reading and mathematics achievement tests. 
EVALUATION METHODOLOGY 

Project impact was assessed by analysis of student 
performance on project developed reading and mathematics tests. 
Two forms were prepared for each test: one for third and fourth 
graders (see Appendix A) and another for fifth and sixth graders 
(see Appendix B) . The tests were administered on a pretest and 
posttest basis in November 1988 and May 1989, respectively. 
FINDINGS 

Pre- and posttest scores for 37 students were submitted for 
evaluation. Table 1 shows student performance on the reading 
test, by grade. Mean pretest raw scores ranged from 6.7 to 9.3 
poinic, mean posttest raw scores ranged from 12.7 to 16.3 points, 
and mean gains ranged from 6.0 to 7.3 points. Fifth graders 
achieved the largest mean gain (7.3 raw points or 36.5 percent 
increase) . 

Table 2 presents students' mean scores on the mathematics 
test, by grade. Mean pretest raw scores ranged from 4.5 to 

2 
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TABLE 1 

Students' Mean Raw Scores* on Project ••Developed 
Reading Tests,* by Grade 
Students-Parents as Partners, 1988-89 



Pretest Mean Posttest Mean Mean Gain 

Raw Percent Raw Percent Raw Percent 



Grade 


N 


Score 


Correct 


Score 


Correct 


Score 


Correct 


3 


19 


6.7 


33.5% 


12.7 


63.5% 


S.O 


30.0% 


4 


7 


9.3 


46.5 


15.7 


78.5 


6.4 


32.0 


5 


7 


9.0 


45.0 


16.3 


81.5 • 


7.3 


36.5 


6 


4 


8.0 


40.0 


14.7 


73.5 


6.7 


33.5 


Total 


j7 


7.8 


39 0 


14.2 


71.0 


6.4 


32.0 



*The Reading test consists of two different forms; one for 
grades three and four; another for grades five and six. Perfect 
raw score ^ for each test = 20. 



Mean gains ranged from 30 to 36.5 percentage points. 



3 

45 



93422 



TABLE 2 

Students' Mean Raw Scores* on Program-Developed 
Mathematics Tests /by Grade 
Students-Parents as Partners, 1988-89 







Pretest Mean 


Posttest Mean 


Mean Gain 


Grade 


Raw Percent 
N Score Correct 


Raw Percent 
Score Correct 


Raw Percent 
Score Correct 



3 


19 


4.5 


22.5% 


9.8 


49.0% 


5.3 


26.5% 


4 


7 


7.3 


36.5 


13 .9 


69.5 


6.6 


33.0 


5 


7 


6.3 


31.5 


13.0 


65.0 


6.7 


33.5 


6 


4 


5.8 


29.0 


9.8 


49.0 


4.0 


20.0 


Total 


37 


5.5 


27.5 


11.2 


56.0 


5.7 


28.5 



'The Mathematics test consists of two different fomns; one for 
grades three and four; another for grades five and six. Perfect 
raw score for each test =20. 

Sixth graders with complete test information showed a 
25.5 percent gain. 
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7.3 score points; mean posttest raw score ranged from 9.8 to 13.9 
score points ; with mean gains ranging from 5.3 to 6.7 score 
points. Fifth graders again achieved the largest mean gain (33.5 
percent increase) . 

Table 3 reports the percentage of students achieving a 30 
percent increase at posttest on both the reading and mathematics 
tests ; by grade. Overall, 59.5 percent of the participants on 
the reading t^^t, and 43.2 percent of the participants on the 
mathematics test, met the project-set criterion for success. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Students-Parents as Partners Project was not 
successful in meeting its objective of 75 percent of participants 
achieving a 30 percent increase in mathematics and language arts. 
Overall; only 59.5 percent of the participants on the reading 
test, and 43.2 percent of the participants on the mathematics 
test, met the project-set criterion for success. Although the 
project was not judged to be successful by the present objective, 
it did appear to be having a positive impact on participating 
students. All students made positive mean gains, with some 
students receiving mean gains as high as 12 score points (60 
percent) . 

It would appear that the project is having a positive effect 
on students* reading and mathematics skills, but that the present 
objective is too ambitious. Therefore, it is recommended that 
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TABLE 3 

Percentage of Students Meeting Project-Set Criterion*, 

by Grade 

Students-Parents as Partners, 1988-89 



Meeting Criterion 

Reading Mathematics 
Grade N N % N % 



3 


19 


11 


58.9% 


7 


36.8% 


4 


7 


5 


71.4 


4 


57.1 


5 


7 


4 


57. 1 


4 


57. 1 


6 


4 


2 


50.0 


1 


25.0 


Total 


37 


22 


59.5 


16 


43.2 



*Seventy-f ive percent of participants will demonstrate a 30 
percent increase in mathematics and language arts. 

overall, 59.5 percent of participating students met the 
project-set criterion for r>uccess on the Reading test 
and 43.2 percent on the mathematics test. 
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the project objective be revised. The objective should be 
revised to state "60 percent of participating students will make 
a 20 percent increase." 



STUDENTS - PARENTS AS PARTNERS APPENDIX A 

P«S. 54K 93422 
MATHEMATICS (GRADES 3-4) 



NAME: DATE: 



1. 73 A) 87 B) 97 C) 60 D) 61 E) NG 
+14 

2. 29 A) 17 B) 511 C) 57 D) 61 E) NG 
+ 32 

3. 14 + 29 = 



A) 15 B) 43 C) 313 D) 163 E) NG 

4. 452 A) 803 B) 8,103 C) 001 D) 903 E) NG 

+451 



5. 1471 A) 3258 B) 16,910 C) 2,690 D) 2,700 E) NG 
+1229 



6. 56 A) 48 B) 44 C) 34 D) 68 E) NG 

-12 



7. 71 - 19 = 



A) 52 B) 68 C) 90 D) 62 E) NG 

8. 783 A) 943 5) 620 C) 173 D) 623 E) NG 

+160 



9. 117- 23 = 

A) 140 B) 84 C) 147 D) 347 E) NG 

10. 402 A) 593 B) 211 C) 891 D) 391 E) NG 
+191 



11. 4 ONES AND 5 TENS = 

A) 9 B) 54 C) 20 D) 1 E) NG 

12. 3 ONES 8 HUNDREDS AND 6 TENS = 

A) 17 B) 386 C) 683 D) 863 E) NG 

13. SEVEN THOUSAND ONE HUNDRED = 

A) 7,001 B) 7,100 C) 70,100 D) 7,010 E) NG 

14. THE GREATEST NUMBER SHOWN BY USING THE DIGITS 1, 7, 3, ONLY ONCE 
EACH IS: 
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15. 32 TENS «• 



15. PETER KAS 43 CENTS. HE SPENDS 8 CENTS. 
KOW MUCH DOES HE HAVE LEFT? 



17. JIM TOOK 2 PENS lis FROM A BOX OF 1 DOZEN. 
HOW MANY PENCILS WERE LEFT? 



18. WHICH OF THESE NUMBERS IS THE LARGEST? 

A) 923 B) 892 C) 907 D) ^89 

19. COMPLETE THE PATTERN 

21, 24, ■ , 30, 



20. COMPLETE THE PATTERN 
41, 39, . , 
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SCIENCE ENRICHMENT PROGRAM, 1988-89 

School-Community Education Program 
Program Administrator: M, Morris Speiser 
Project Coordinator: Herbert Ross 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Science Enrichment program provides after-school science 
instruction to elementary schoox students in Community School 
District (C.S.D.) 23. Recognizing the need to improve the level 
of student academic achievement in science, the project, seeks to 
supplement and enrich regular science courses • In 1988-89, the 
project served 420 students in grades three through six in 14 
elementary schools. Teachers and school principals selected 
students according to their need for the program and their 
willingness to participate in the program. 

Students attended two-hour after-school sessions two days 
per week. Classroom instruction focused on electricity, 
friction, gravity and motion, the solar system, weather and 
climate, water, basic chemistry and physics, and the environment. 
To encourage the development of pupil reasoning and thinking 
skills, the program emphasized hands-on instruction, pupi3 
experimentation, and problem-solving activities. The Silver- 
Burdett Master Equipment Package served both to train teachers 
and to involve students in hands-on and laboratory-like 
instructional experiences. In 1988-89, the project had two 
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objectives: sixty percent of participating students to increase 
their knowledge of science concepts and information by 20 
percent, as measured by a project-developed test; and seventy 
percent of participating teachers to receive a rating of 20 
points or more on a mastery checklist. 

C.S.D. 23 staff members and the district science coordinator 
supervised the program, and licensed teachers provided classroom 
instruction. The New York State Legislature contributed $44 
thousand to pay for teachers* after-school instructional 
activities, and to purchase the Silver-Burdett Master Equipment 
Package and teacher ^.raining tapes. 
EVALUATION METHODOLOGY 

Evaluation activities for the first objective focused on the 
analysis of student perforinance on a project-developed test (see 
Appendix A) . The test consists of 25 multiple-choice items on 
conceptual and factual science knowledge. The test was 
administered both as a pretest and posttest at the beginning and 
end of program activities. 

The second objective was analyzed through teachers scores on 
a. district-developed mastery checklist (see Appendix B) . 
Teachers were rated on six areas designed to measure their 
effectiveness in implementing the program. Maximum possible 
score was 30 points. 
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FINDINGS 

For the first objective, data was received for 311 students 
from 13 schools for whom valid pre- and posttest scores were 
available (see Table 1). Overall, mean pretest raw score was 
10.3 points (41.2 percent correct responses), and mean posttest 
raw score was 15.6 points (62.4 percent correct responses), for a 
mean gain of 5.3 points, or 21.2 percent. While mean pretest 
scores by school ranged from 8.6 to 12.4 raw score points, there 
was a wider range of mean posttest scores (from 10.3 to 19 -.5 raw 
Fcore points), and of mean gains (from 0.6 to 8.8 raw score 
points). Pupils from P.S. 178 made the largest mean gains* 

Wheii test scores were analyzed individually, student 
performance showed a wide range of variability (see Table 2) . 
Overall, 57.2 percent of participants increased their knowledge 
of science concepts and information by 2 0 percent. By school, 
students from P.S 41, 150, 156, 165, 178, 183, and 284 met or 
surpassed the project-set criterion. 

For the second objective, ratings were received for 25 
teachers on the mastery checklist. Overall mean was 24.4 points 
out of a possible 30. Twenty-one (84.4 percent) of the teachers 
met or surpassed the project-set criterion of receiving a rating 
score of 20 on the implementation of the program. Only four 
teachers did not meet this criterion, and all four of these 
teachers received a rating of 19. 
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TABLE 1 

Students' Mean Raw Scores' on a Project-Developed Test, 

By School 
Science Enrichment Program, 1988-89 







Pretest Mean 


Posttest Mean 


Mean Gain 


School 




Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


P.S . 41 


21 


9.8 


39.2% 


15.3 


61.2% 


5.5 


22.0% 


73 


33 


8.6 


34.4 


11.8 


47.2 


3.2 


12.8 


137 


1 1 
11 


10.2 


40.8 


14.1 


56.4 


3.9 


15.6 


150 


21 


9.9 


39.6 


14.9 


59.6 


5.0 


20.0 


156 


14 


11.5 


46.0 


15.9 


63.6 


4.4 


17.6 


165 


30 


11.8 


47.2 


19.5 


78.0 


7.7 


30.8 


178 


32 


10.5 


42.0 


19.3 


77.2 


8.8 


35.2 


183 


13 


11.2 


44.8 


17.3 


69.2 


6.1 


24.4 


184 


31 


12.4 


49.6 


16.5 


66.0 


4.1 


16.4 


284 


30 


9.5 


38.0 


16.8 


67.2 


7.3 


29.2 


298 


38 


10.4 


41.6 


15.3 


61.2 


4.9 


19.6 


327 


12 


8.6 


34.4 


13.5 


54.0 


4.9 


19.6 


332 


25 


9.7 


38.8 


10.3 


41.2 


0.6 


2.4 


TOTAL 


311 


10.3 


41.2 


15.6 


62.4 


5.3 


21.2 


"Perfect 


Raw 


Score=25 












Overall, 


mean gain 


was 21.2 


percent. 









Students at P.S. 178 achieved the largest mean gain. 



TABLE 2 

Percentage of Participants Meeting Project-Set Criterion/ 

by School 
Science Enrichment Program; 1988-89 







MeetiDQ 


Criterion 


Schoo.l 


N 


N 


% 


P.S. 41 


21 


14 


66.7% 


73 


32 


12 


36.4 


137 


11 


3 


27.3 


150 


21 


13 


61.6 


156 


14 


9 


64.3 


165 


30 


26 


86.'/ 


178 


32 


25 


78.1 


183 


13 


9 


69.2 


184 


31 


11 


33.3 


284 


30 


26 


86.7 


298 


38 


22 


57.9 


327 


12 


5 


41.7 


332 


25 


3 


12.0 


TOTAT 


311 


178 


57.2 



•sixty percent of participants will increase their knowledge of 
science concepts and information by 20 percent. 



* Overall; 57.2 percent of participants met the project- 
set criterion for success. 
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CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Science Enrichment Program succeeded in 
meeting one of its objectives and was partially successful in 
meeting the other. For the first objective, the evaluation 
findings show that the project was partially successful. While 
students at all schools increased their knowledge of science 
concepts and information, this improvement varied considerably 
between schools and between students. Overall, mean gain was 
21.2 percent but mean gains by school ranged from 2.4 to 35.2 
percent. The percentage of students who met the project-set 
criterion for success by school showed a wide range of 
variability. Students from 7 of the 13 schools met the project- 
set criterion. In the rest of the schools, this percentage 
ranged from 12 to 58 percent of successful students. The overall 
percentage of students who met or surpassed the project-set 
criterion was 57.2 percent rather than 60 percent. 

The second objective was for 70 percent of teachers to 
receive a rating of at least 20 on their effectiveness in 
implementing the program, in 1988-89, the Science Enrichment 
program was successful in meeting this objective. Overall, 84.4 
percent of the teachers met or surpassed the project-set 
criterion. Only four teachers did not meet this criterion and 
all four of these teachers received ratings of 19 instead of 20. 
Two of these four teachers taught at P.S. 178, although this 
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school displayed one of the highest rates of student success (78 
percent) . 

With the information provided to the Office of Research, 
Evaluation, and Assessment, it is not possible to determine why 
some schools performed better than others. The participating 
teachers all received high ratings on their implementation of the 
project, yet only half of the participating schools' students met 
the project-set criterion. Those students who are in real need 
of remedial instruction may require more individualized or small 
group instruction. In order to improve the impact of the project 
on student growth; project staff may wish to emphasize small 
group or individualized instruction. 
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PRE- TEST AND POST-TEST FOR THE 
SCnHS ENRICHMENT PROGRAM 
(PROGRAM 93401 (#25) 

School: Grade:_ 

Student's Naine: Date: 



i^ctions: Circle the answer for each question. 
1. A liter is used to measure 

a) length 

b) volume 

c) mass 

d) tenperature 

2. If you wanted to measure the length of your pencil, you would us( 

a) centimeters 

b) meters 

c) kilometers 

d) grams 

3. If you were asked to measure vAich is heavier, a crayon 
or a pencil, you would use a 

a) ruler 

b) scale 

c) thermometer 

d) graduated cylinder 

4. In the metric system, teirperature is measured in 

a) Fahrenheit 

b) Celsius 

c) millimeters 

d) centigrams 

5. When reading the thermometer, you should 

a) hold it at the top 

b) read it at eye level 

c) keep it in the material vAose temperature you 
are reading 

d) all of the above 

6- During photosynthesis a plant needs all of the following except 

a) chlorophyll 

b) carbon dioxide 

c) soil 

d) sunlight 

59 



7. In wtiat part of the plant does photosynthesis take place 

a) root 

b) stem 

c) leaf 

d) flower 

8. The primary function of a flower is 

a) reproduction 

b) photosynthesis 

c) absorption of water and minerals 

d) passageway for food and water. 

9 An example of a stem is 

a) broccoli 

b) celery 

c) a potato 

d) a carrot 

10. All of v.e following are exanples of parts of a plant except 

a) pupa 

b) sepal 

c) pistil 

d) stem 

11. Seeds are spread by 

a) wind 

b) birds 

c) water 

d) all of the above 



12. Caterpillars become 

a) flies 

b) spiders 

c) butterflies 

d) beetles 

13. If an animal has 6 legs, 3 body parts, 2 pairs of wings 
and 2 antennae it is 

a) an insect 

b) a fish 

c) a bird 

d) a spider 
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14. All of the following pre exaiiples of stages in the life 
butterfly except 



a) egg 

b) pupa 

c) larva 

d) sepal 

15. Which of the following is not part of a seed 

a) enfcryo 

b) stigma 

c) seed coat 

d) cotyledon 

16. Electricity flows along a path called a 

a) circuit 

b) cable 

c) battery 

d) conductor 

17. To open or close an electrical circuit you could use a 

a) bulb 

b) light 

c) switch 

d) watt 



18. Magnets attract 

a) sil/er things 

b) iron and steel 

c) aluminum 

d) wood 

19. Electrical wires are usually made of copper because coj^r i 

a) a good insulator 

b) shiny 

c) a heavy material 

d) a good conductor 

20. All of the following will be attracted to a magnet 
except a 

a) nail 

b) penny 

c) cork 

d) paperclip 
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21. If you use your five senses to learn about something, 
you are using the process skill of 

a) prediction 

b) observation 

c) classifying 

d) measuring 

22. If you take a set of objects and group them into those 

that are attracted to a magnet and those that are not attracted 
to a magnet, you are using the process skill of 

a) inferring 

b) communicating 

c) classifying 

d) formulating a hypothesis 

23. If you first take the temperature of two cups of water, and then 
make a statement about what will happen to the water's tenperature 
if you mix them together, you are making a 

a) prediction 

b) observation 

c) measurement 

d) classification 

24. While performing an experiment about pendulums, you make the ■ 
following statement; ^ If I shorten the string on the pendulum, 
then the number of swings will increase, ihis is an example of a 

a) classification 

b) hypothesis 

c) measurement 

d) prediction 

25. If you pour 30 ml. of water into a graduated cylinder, you are 
using the process skill of 

a) communicating 

b) measuring 

c) predicting 

d) classifying 



END END END END 
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COMPUTER LITERACY PROGRAM, 1988-89 

School-Comiaunity Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinators: Joe Pawlis 

C. Matkowsky 
S. Cooper 
Shelley Leibson 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Computer Literacy program is designed to provide 
students with basic computer knowledge and to expose them to 
career opportunities in the field of computers. In 1988-89, the 
program served some 800 students in grades two through nine at 
Community School Districts (C.S.D.s) 11, 12, 27, and 28. School 
principals selected students for participation in the program* 
Students attended 45-minute sessions two or three days a 
week. Classes were held in model computer centers containing 
between five and ten computers and appropriate software. Class 
activities focused on computer languages , use of the equipment, 
and work with pre-programmed Computer Assisted Instruction 
programs in various curricula areas. 

The objective of the program was for 75 percent of the 
participants to demonstrate a 30 percent increase in the 
understanding of computer literacy by composing and executing 
simple programs, executing ba^ic computer commands, and 
identifying careers related to the computer field. Student 

83 
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growth was measured by pre- and posttest outcomes on a project- 
developed test. 

Project staff consisted of tax-levy teachers already working 
in the participating schools, and school principals who 
supervised all program activities. The New York State 
Legislature contributed $110 thousand to pay primarily for 
computer equipment and instructional supplies, 
EVALUATION METHODOLOGY 

Evaluation activities focused on analysis of student 
performance on a project-developed test (see Appendix A) . The 
test consisted of five separate sections. The first section 
contained 15 multiple-choice items on basic computer uses, terms, 
and applications. Each of the other four sections contained 10 
multiple-choice items on specific programming languages or 
software such as LOGO, word processing, basic programming and 
data base. All districts were required to complete the first 
section and each district had to complete two of the other four 
sections which best reflected the emphasis of the program in that 
district. The test was administered at the beginning and end of 
project activities* 
FINDINGS 

Table 1 reports the progress of 156 students from three 
districts for whom valid pretest and posttest scores were 
available. All of the districts completed the general and 
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TABLE 1 

Students' Mean Raw Scores* on a Program-Developed Test, 

By District 
Computer Literacy Program, 1988-89 







Pretest Mean 


Posttest Mean 


Mean Gain 


District'' 


N 


Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


11 


49 


21.8 


62.3% 


25.8 


73.7% 


4.0 


11.4% 


12 


80 


19.2 


54.6 


28.3 


80.9 


9.1 


25.7 


27 


27 


14 .6 


41.7 


18.6 


53.1 


4.0 


11.4 


TOTAL 


156 


19.3 


55.1 


25.7 


73.4 


6.6 


18.9 



^Perfect Raw Score=35 

'^All C.S.D.s completed the general section and the word 
processing section of the test. In addition, C.S.D* 11 completed 
the Basic programming section; C.S.D. 12 the LOGO section; and 
C.S.D. 27, the Data Base section. 



Overall, mean gain was 6.6 raw score points or 18.9 
percentage points. 
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the word processing sections of the test. Each district 
completed one additional section of the test for an overall 
maximum score of 35 points: C.S.D. 11 completed the basic 
programming section; C.S.D. 12, the LOGO section; and C.S.D. 27, 
the data base section. Overall, mean pretest raw score was 19.3 
points (55.1 percent correct responses); and mean posttest raw 
score was 25.7 points (73.4 percent correct responses)/ for a 
mean gain of 6.6 points or 18.9 percent. 

Table 2 presents the percentage of students from each 
district who met the project-set criterion of a 30 percent 
increase. Overall; twenty-seven percent of participating 
students met this criterion. District 11 achieved the highest 
success rate; with 50.0 percent of participants meeting the 
project-set criterion. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89; the Computer Literacy program was not successful 
in meeting its objective of 75 percent of the participants 
achieving a 30 percent gain in computer literacy. Overall; only 
27.6 percent of the students met this objective. 

In 1988-89, the testing instrument was revised replacing one 
general test with the current version utilizing the various 
sections. This was done to accommodate the individual needs of 
each district. There was a variation in scores across districts. 
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TABLE 2 

Percentage of Students Meeting Project-Set Criterion,' 

by District 
Computer Literacy Prograia, 1988-89 









Meetina .Criterion 


District 


N 


N 


% 


11 


49 


1 


2.0% 


12 


80 


40 


50.0 


27 


27 


2 


7.4 


Total 


156 


43 


27.6 



®Seventy-f ive percent of participants will achieve a gain of at 
least 30 percent from pretest to posttest. 



Twenty-seven percent of participants met the project- 
set criterion for success. 
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It is difficult to assess what could have caused this difference* 
One possible explanation is a variation in the difficulty levels 
of each section of the test. For example, students scored the 
lowest mean gain on the data base section of the test and the 
largest mean gain on the LOGO section of the test. Because those 
students who took the data base secLion did not also take the 
LOGO section, it is impossible to adequately assess whether these 
differences are a function of the testing instrument or 
variations in teaching or students abilities across districts. 

All of the students completed the general section of the 
test on basic computer uses, terms, and applications. Mean 
pretest scores by district ranged from 9 to 13 score points out 
of a maximum of 15 possible score points. Such high pretest 
scores preclude any improvement on posttest (ceiling effect) and 
indicate that this section of the test is too easy. It is 
I'ecommended that this section of the test be revised to 
increase the difficulty level of the questions. 

Another possible explanation for the variation of scores 
across districts is the grade level of the students: C.S.D. 11 
reported scores for students in grades 5 to 9, C.S.D. 12 reported 
scores for students in grade seven, and C.S.D. 27 reported scores 
for students in grade four. 

In 1987-88, complete test scores were reported for 749 
students. This year, data was reported for only 156 students. 



erJc 



93433 



Much of the data that was received was incomplete or incorrectly 
filled out. In the future ; an effort should be made to provide a 
complete set of scores for all students participating. 



APPENDIX A 

GZ:-ERA1 COMPUTER LITERACY - MODCLE I 93433 
PRE - POST^-TEST 
(B/E #5001-48-834"33) 

Directions ; Enter your answers on a separate answer sheet. 

Do not leave any blanks on your answer sheet. 

1. This part of a computer systeia is the: 




(a) disk drive. 

(b) monitor . 

(c) printer • 

(d) keyboard, 

(e) abacus. 



2. This part of a computer system is the: 




(a) monitor. 

(b) printer. 

(c) keyboard . 

(d) scanner • 

(e) disk drive, 



3. This part of a computer system is tne: 









m 



(a) disk drive, 

(b) monitor . 

(c) scanner . 

(d) keyboard. 

(e) printer . 



4. This part of the computer system is the: 




(a) printer, 

(b) monitor . 

(c) abacus. 

(d) disk drive. 

(e) keyboard . 



5. The name of the symbol that appears on the video screen that 
shows us where the next character to be typed is the; 

(a) disk. 

(b) period. 

(c) cursor. 

(d) semicolon. 

(e) comma. 

-1- 

GO ON 
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ITEM ANALYSIS FOR SASIC PROGRAMMING 



iiOTE: This module consists of a 10 items. 



Statements 
1 (PRINT) 

7 (GOTO) 

8 (LET) 



Programming 
2 (Use of line number) 
5 (Use OF END statement) 
6- (Use of comma and semicolon) 



Commands Calculating 
3 (RUN) 4 



String Variable 
10 
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ANSWER KZYTCR THE BA.' C ^ROGRAMMiNG MODCLE 



Item analysis sheet attacned for copy . 



ANSWER 




TOPICS 


1. 


B 


PRINT Statement 


2. 


C 


Use of a line number 


3. 


B 


RUN Command 


4. 


C 


Output of a program 


5. 


D 


Progranuiiing 


6. 


B 


Programming 


7. 


E 


GOTO Statement 


8. 


A 


LET Statement 

* 


9. 


B 


LET Statement 


10. 


C 


String variable 
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9. 



Tr.e output for this program is: 



iC 


LET C 


= 


20 


PRINT 


C 


30 


LET N 


= 


40 


PRINT 


N 


SO 


END 




RUN 




(a) 


14 






10 




(b) 


14 






24 




(c) 


24 






14 




(d) 


24 




(e) 


14 





10 



The output for this program is: 



10 LET B = 16 

2C LET B$ = "DIVISION" 

30 PRINT B, B$ 

40 END 

RUN 



(a) B, B$ 

(b) DIVISION, 16 

(c) 16 

(d) B, 

(e) 16, 



DIVISION 
DIVISION 
DIVISION 



-4- 



STOP 



STOP 



STOP 



STOP 



STOP 



STOP 



STOP 



STOP STOP 
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T.'-.e o'Jtput: for this program is: 



10 PRINT "I LIKE BURGERS 
15 GOTO 40 

20 PRINT "I LIKE SODA" 
30 PRINT "I LIKE FRIES" 
40 PRINT "I LOVE FOOD" 
50 END 

(a) I LIKE BURGERS 
I LIKE SODA 

I LOVE FOOD 

(b) I LIKE SODA 
I LIKE FRIES 
I LOVE FOOD 

(c) I LIKE BURGERS 
I LOVE FOOD 

I LIKE FRIES 

(d) I LIKE BURGERS 
I LIKE SODA 

I LIKE FRIES 
I LOVE FCOD 

(e) I LIKE BURGERS 
I LOVE FOOD 



8. The output for this program is: 

10 LET X = 8 

20 LET y = 10 

30 LET Z = 12 

40 PRINT X, y , Z 

50 END 

RUN 



(a) 
(b) 
(c) 
(d) 
(e) 



8 10 12 

X y Z 

PRINT X y 
30 



none of the above. 



GO ON 
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What IS wrong with this program? 



10 PRINT "STARS" 
20 PRINT •• APPLES" 
30 PRINT "HOUSE" 
40 PRINT "GAMES" 
50 PRINT "FLOWERS" 
60 PRINT "SKIES" 
RUN 

(a) There are no statements. 

(b) There are no commands. 

(c) There are no periods. 

(d) There is no END statement. 

(e) There is no line number for RUN. 



6. The output for this program is: 

10 PRINT "HOME", "HOTEL" 
20 PRINT "HOME"; "HOTEL" 
30 END 
RUN 



(a) 



HOME HOTEL 



(b) 



HOME 
HOME 



HOTEL 
HOTEL 



HOMEHOTEL 



(c) 



HOME 
HOME 
HOTEL 
HOTEL 



HOTEL 
HOTEL 
HOME 
HOME 



(e) 



HOMEHOTEL 



HOMEHOTEL 



-2- 
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EA5IC PROGRAMMING - MODULE V 
PRE - POST TEST 
(B/E #5001-48-83433) 

Directions ; Enter your answers on a separate sheet. 

Do noc leave any blanks on your answer sheet. 



If you typed this on the screen/ the computer would display: 
PRINT "METS'* 

(a) "METS" 

(b) METS 
(C) STEM 

(d) PRINT METS 

(e) "PRINT METS" 



Each instruction in a program must start with a: 

(a) capital letter and a comma. 

(b) comma and a period, 

(c) line number. 

(d) quotation mark. 

(e) command. 



The RUN command tells the computer to: 

(a) stop working^ 

(b) do something immediately. 

(c) erase a program 

(d) create a line number, 

(e) clear the screen. 



The output for this program: 

10 PRINT 16+4 

20 END 

RUN 

(a) PRINT 20 

(b) 4+16 

(c) 20 

(d) 64 

(e) 16+4 



-1 



GO ON 
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Item; Analysis for the Word Processing Module 

Note ; This Module consists of 10 items 

Meanings Functions 

1 Word Processor 2 (Word Processor) 

3 TAB 9 (Word Processor) 

4 FORMAT 10 (Word Processor) 

5 EDIT 

6 SAVING 

7 DELETE 

8 INSERT 
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ANSWER KEY FOR THE KEY WORD PROCESSING MODULE 



Answer Topics 



1 


c 


Meaning of a Word Processor 


2 


D 


Function of a Word Processor 




B 


Meaning of a TAB 




C 


Meaning of FORMAT 


5. 


A 


Meaning of EDIT 


6. 




Meaning of FILE 


7 • 


A 


Meaning of DELETE 


8. 


E 


Meaning of INSERT 


0 ^ 


B 


Function of a Word Processor 


10. 


C 


Function of a Word Processor 
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10. 



If you find a mistake m a letter while you are prmtina, you 
are : 

(a) unable to correct the mistake by using the word 
processing program. 

(b) unable to stop the printing to correct the mistake. 

(c) able to stop the printing and correct the mistake. 

(d) able to stop the printing, but unable to correct the 
mistake • 

(e) able to check for more mistakes, but unable to 
correct them. 



-3- 
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When you edit using a word processing program, you can: 

(a) correct mistakes. 

(b) set the margins on the paper. 

(c) identify tne number of lines you need. 

(d) identify the type of print you need. 

(e) set the number of words on a page. 



6» Saving a file means: 

(a) erasing it from the computer's memory. 

(b) correcting mistakes in a file. 

(c) formating a file. 

(d) using a word processing program. 

(e) keeping the information on a disk. 



7. When you delete something, it means you: 

(a) remove it. 

(b) add to it. 

(c) copy it. 

(d) underline it. 

(e) save it. 

8. When you insert something in a paragraph, you: 

(a) save it. 

(b) remove it. 

(c) erase it. 

(d) underline it. 

(e) add it. 



9. If you wanted a person to notice an important sentence in a 

letter, you could use the word processing program to: 

(a) copy the important sentence throughout the letter. 

(b) underline the sentence where it appears in the 
letter . 

(c) print only the important part of the letter and leave 
out the remaining part of the letter. 

(d) increase the speed in which the letter is printed. 

(e) none of the above. 
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Directions : Enter your answers on a separate answer sheet. 

Do not leave any blanks on your answer sheet. 

The best definition of a word processor is: 



(a) 


a computer program that 
work but not correct it, 


allows 

» 


people 


to 


type their 


(b) 


a computer program that 
thexr work but not save 


allows 
it. 


people 


to 


complete 


(c) 


a computer program that 
correct their work. 


allows 


people 


to 


write and 


(d) 


a computer program that 
their work with others. 


allows 


people 


to 


compare 


(e) 


none of the above . 











A word processing program can be used to do all these iobs 
EXCEPT: 

(a) show all of the work before it is sent to a printer. 

(b) correct the errors on the screen. 

(c) move words and paragraphs around on the screen. 

(d) write a letter without the input of a person. 

(e) print more than une copy of a letter without 
retyping it. 



To save time when you have to begin a rew paragraph in a 
letter with 6 paragraphs, you can: 

(a) set an index for each new paragraph. 

(b) set a tab to be used for each new paragraph. 

(c) set a page margin for each new paragraph. 

(d) set the cursor using the spacebar each time you must 
begin a new paragraph. 

(e) set the automatic display key to six. 



When you format using a word processing program, you can: 

(a) edit mistakes on a letter. 

(b) complete difficult math problems. 

(c) set the margins. 

(d) create tables and graphs. 

(e) type out a finished letter. 



-1 
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Ite.T>; Analysis for Data Base Module 
NOTE ; This module consists of 10 items. 

Meanings Procedures 

1 (data base) 5 

2 (field) 7 

3 (retrieval) 8 

4 (menu) 
9 (SORT) 

10 (RETRIEVAL) 



use of a Data Base 



8 



93433 



ANSWER KEY FOR THE DATA BASE MODULE 



ANSWER TOPICS 



1 . 


C 


2. 


D 


3. 


B 


4 . 


A 


D • 


D 


6. 


C 


7. 


E 


8. 


C 


9. 


B 


10. 


A 
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Meaning of Data Base 
Meaning of a field 
Meaning of retrieval 
Meaning of of menu 
Procedures for a data base 
Use of a Data Base 
Procedures for a D-^ta Be^e 
Procedures for a Data Base 
Meaning of SORT 
Meaning of RETRIEVAL 
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6. A teacher could use a data base for: 

(a) constructing a maze* 

(b) writing a Ic-tcsr. 

(c) listing tho addresses of children. 

(d) editing a letter. 

(e) marking a test. 

7. When you enter information into your data base, you must: 

(a' complete more than one record at a time. 

(b) print it br'"ore you complete it. 

(c) remove the name of the field. 

(d) delete all the information in the field 

(e) complete one record at a time. 

« 

8. Before you can use a disk to save your data base file, you 
must : 

(a) erase the entire field. 

lb) use the TAB key. 

{c) initialize the disk. 

(d) create a new field. 

(e) none of these. 



If teachers wanted to examine the "last names" of their 
students* data base record-^/ they must: 

(a) rename their records. 

(b) save their records. 

(c) edit their records. 

(d) delete their records. 

(e) sort their records. 



10, 



If you have forgotten the name of your data base file, which 
of these would you need to retrieve: 

(a) a list of files. 

(b) a new menu of commands. 

(C) a list of in«?truction for saving your file 

(d) a set of directions for creating a new file. 

(e) a menu for deleting your files. 



-2- 
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Directions 



DATE BASE ~ MODULE IT I 
PRE - POST»>T£ST 
(3/E #5001-48-83433) 

Enter your answers on a separate sh,^et. 

Do not leave any blanks on your ansver sheet, 



1. A data base program helps people to: 

(a) organize and save stories. 

(b) create pictures and drawings for books. 

(c) organize and save information. 

(d) create games and puzzles. 

(e) print long stories without pictures. 

2. To ^nake a new data base you must: 

(a) create bases. 

(b) create filers. 

(c) crea e tabs. 

(d) create fields. 

(e) create drives. 



If you wish to work again on an old file/ you must do the 
following : 

(a) highlight it. 

(b) retrieve it. 

(c) underline it. 

(d) delete it. 

(e) position it. 



A MENU is a; 



(a) ]ist of things a program can do, 

(b) list of files on a disk. 

(c) list of fields on a screen, 
(a) list of records on a disk. 

(e) list of aata on a storage disk. 



After you have created fields # you can now: 

(a) remove the information. 

(b) sort the information. 

(c) count the information. 

(d) enter the information. 

(e) review the information. 



-1- 
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ITEM; Analysis for the LOGO Module 
NOTE: This Module consists of 10 items. 



Knowledge of Primitives 



1 
2 



(LT and l.D 



Meaning of Commands 

3 (PU or PENUP) 

4 (PD or PENDOWN) 



Knowledge of Geometry 
5 (triangle) 



Knowledge of LOGO 

6 (spacing) 

7 (HOME Command) 

8 (REPEAT Command) 

9 (Draw a square) 
10 (FD 40 Command) 
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ANSWER KEY FOR TEE LOGO MODULE 



ANSWER TOPICS 

!• D Knowledge of Function Keys 

2. C Knowledge of Function Keys 

3. A Meaning of Commands 

4. E Meaning of Commands 

5 D Knowledge of Geometry 

6. D Knowledge of LOGO 

7. E Knowledge of LOGO 

8. A Knowledge of LOGO 

9. B Knowledge of LOGO 
10. C Knowledge of LOGO 



ERLC 
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6. Following FD, you must leave a before typing how 

many steps the c 'rtle should move: 

(a) comma. 

(b) period* 

(c) zero. 

(d) space. 

(e) colon 

7. One way to return the turtle to the center of the screen is 
to: 

(a) clear the screen using the EDIT key. 

(b) erase the lines using the PE command, 
. (c) use the RETURN key more than once. 

(d) move the turtle by using the BACK command. 

(e) type HOME. 

S. When you use the REPEAT command in a LOGO program: 

(a) the commands between the brackets need to be typed. 

(b> there is no longer a need for commands. 

(c) commands need to be written more than once. 

(d) there is no longer a need for numbers or directions. 

(e) the turtle draws lines and erases them. 

9. 



The commands 


below will 


draw this 


TO SURPRISE: 






FD 3 0 






RT 90 


(a) 


circlfi 


FD 30 


(b) 


squar e . 


RT 90 


(c) 


tr iangle . 


FD 30 


(d) 


cube . 


RT 90 


(e) 


rectangle. 


FD 30 






RT 30 






END 






The command 


FD 40 makes 


the turtle 



10. 

(a) move downward 40 turtle steps. 

(b) move upside down 40 turtle steps. 

(c) move forward 40 turtle steps. 

(d) move backward 40 turtle steps. 

(e) move upward 40 turtle steps. 

STOP STOP STOP STOP STOP 
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LOGO - MODULE II 
PRE - POST TEST 
(B/E #5001-48-83433) 

Directions : Enter your answers on a separate answer sheet. 

Do not leave any blanks on your answer sheet. 

If the pen is down, all of these coanands would make the turtle 
draw a line EXCEPT: 

(o) FD 75 

(b) FD 40 

(c) BK 300 

(d) LT 85 

(e) BK 2 



The two comiTjands that make the turtle TT^RN are: 

(a) UP 80? ^OWN 20 

(b) FD 50? 5K 80 

(c) LT 40? RT 70 

(d) CS ? PD 

(e) HOME ? PU 



This command allows you to luove the turtle to a new location 
on the screen without drawing a line; 

(a) PU or PENUP. 

(b) DP or DOWNP. 

(c) UP or ?ENP. 

(d) PD or PENDOWN. 

(e) CS or CLRSC. 



This command PD or PENDOWN allows the turtle to: 

(a) stop drawing and turn. 

(b) clear the screen. 

(c) move to a new location and draw a line. 

(d) move only backward. 

(e) get rer ]y to draw again. 



Figure "x" is an example of a: 

(a) square. 

(b) rectancjle. 

(c) circle. 

(d) triangle . 

(e) cube. 

-1- 
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Items Analysis for tne Core Curriculum 
NOTE ; This module consists of 15 items. 

Computer Parts 

1. Keyboard 

2. Printer 

3. Monitor 

4. Disk drive 



Computer Care 
7 
12 



Computer Commands 
13 



Flowchart 
15 



SO 



Meanings 

5. Cursor 

6 . Software 

8. Input device 

9. BASIC 



Computer Usage 

10 (supermarket) 

11 (legal use) 



Compute r History 
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ANSWER KEY FOR THE GENERAL COMPUTER LITERACY MODULE 



1. D Computer parts (Keyboard) 

2. B Computer parts (printer) 

3. B Computer parts (monitor) 

4. D Computer parts (disk drive) 

5. C Meaning of cursor 

6. E Meaning of Software 

7. D Computer care 

8. B Meaning of input device 

9. A Meaning of BASIC 

10. B Computer usage 

11. E Computer usage 

12. D Computer usage 

13. E Meaning of Computer Commands 

14. A Computer History 

15. E F^Jowchart 
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The step that is INCORRECT when working with a computer is: 

(a) putting^ a disk in a disk drive and loading a program. 

(b) turning on disk drive/ a monitor, and a keyboard 
before you begin to work. 

(c) putting a floppy disk in a disk drire before it is 
turned on, 

(d) eating at the computer. 

(e) turning off a computer after you have saved your work. 



13. If you want to move the cursor FORWARD, you must; 

(a) press the DELETE key. 

(b) press the LEFT ARROW key. 

(c) press the ESCAPE key. 

(d) press the CLEAR SCREEN key. 

(e) press the RIGHT ARROW key. 



14. Blaise Pascal got the idea for his calculator from this 
machine : 

(a) abacus. 

(b) water clock. 

(c) sewing machine. 

(d) printing press. 

(e) piano. 



15. Below is a flowchart to make a bologna s^^indwich with lettuce 



ana mayonnaise 



Ctt 2 tlicts 
of bread. 

and vuttard. 



Which step in the flowchart is missing? 



3>t knift to 
sprtad 

cayonnaitt oo 
brtad . 



V«t knift to 
cut lcttuc« 
and tomato. 



Put tvo pitcts 
of tht brtad 
togtthtr and 
tat* 



Add l«ttuc« 
and tonato 

to the br^ad. 



(a) removing the bologna from the sandwich. 

(b) adding the mayonnaise to the sandwich. 

(c) removing the tomato and the lettuce. 

(d) adding a tomato to the sandwich. 

(e) adding bologna to the sandwich. 



STOP 



STOP 
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6. Ar. exa.T.ple of software is a: 

(a) kevDoard with 32 letters. 

(b) disk drive with a hard disk- 

(c) scanner reader used in banks. 

(d) color monitor. 

(e) storage device containing computer programs. 

7. In order to prevent damage to floppy disks, people should: 

(a) store them upright without a dustproof container. 

(b) always touch the recording surface of the disks. 

(c) store them in a warm place that is damp. 

(d) place them in protective envelopes when they are not 
being used. 

(e) none of the above. 

8. All the following are examples of input devices EXCEPT: 

(a) cassette tape recorder. 

(b) printer. 

(c) disk drive. 
(g) card r*=ader. 
(e) keyboard. 

9. BASIC is a: 

(a) computer language. 

(b) number system. 

(c) special type of keyboard. 

(d) disk with no data. 

(e) part of the computer used to store memory. 

10. A computer is used in a supermarket to: 

(a) hand deliver groceries. 

(b) compute a person's final bill. 

(c) hand mark prices on canned goods. 

(d) weigh and cook fresh food. 

(e) measure the height, weight and age of customers. 

11. Business computers can be used legally to do all the 
following EXCEPT: 

(a) word processing. 

(b) doing inventory. 

(c) electronic mail. 

(d) keeping a record of messages. 

(e) changing the records of other companies. 

-2- 
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#5001-48-93435 

REINFORCING READING; 1988-1989 

School-Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Linore Lindy 

Prepared by: 
Office of Research; Evaluation; and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The goal of Reinforcing Reading is to expose students to 
diversified communications arts activities designed to enrich 
their regular classroom experiences; and resulting in improved 
self-image; enhanced self-esteem and increased reading success. 
The New York State Legislature provided $22 thousand in funding 
fox- this project. 

Two hundred students in grades three through six 
participated in after-school workshops. Students were selected 
for participation in rhe program based upon their test 
performance; school records; and attendance. Participating 
students were screened to determine their needs in reading; 
writing; attitude toward school and attendance. Student needs 
were determined through informal diagnostic testing and student- 
teacher conferences. 

After-school workr/r.ops were held at five elementary schools 
in Community School District (C.S.D.) 16. Each site held two 
after -school classes; one for third and fourth grade students, 
and another for fifth and sixth grade students. Students 
received individualized remedial services based on their 
diagnostic information. Each student received at least foux--and- 

94 



93435 

-a-half hours of remedial assistance per week. Staff consisted 
of licensed teachers selected by the district, and the project 
coordinator, who provided supportive assistance. 

The objective was for participating students to achieve a 
statistically significant mean c"^in of seven Normal Curve 
Equivalent (N.C.E.)* score points from pretest to posttest on the 
Degrees of Reading Power (D.R.P.) Test. 
EVALUATION METHODOLOGY AND FINDINGS 

Project impact was assessed by analyses of students' scores 
on the Degrees of Reading Power (D.R.P.) Test administered in the 
spring of 1988 and the spring of 1989. Raw scores were converted 
to normal curve equivalent (N.C.E.) scores, which describe 
student performance relative to national norms. Mean N.C.E. 
scores were compared, and correlated t- tests were computed to 
establish if achievement differences were statistically 



*N.C.E. scores are similar to percentile ranks, but unlike 
percentile ranks, are based on an equal-interval scale. Normal 
curve equivalent scores are based on a scale ranging from 1 to 99 
with a mean of 50 and a standaj^d deviation of approximately 21. 
Because N.C.E. scores are equf*lly spaced apart , arithmetic and 
statistical calculations such as averaging are meaningful; in 
ac^dition, comparisons of N.C.E, scores may be made across different 
achievement tests. 
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significant. An effect size (E.S-)*/ which indicates the 
educational meaningfulness of the mean gain or loss for each 
comparison was also calculated. 

An evaluation of this year's project was not possible 
because no data was received by the Office of Research Evaluation 
and Assessment. 

CONCLUSIONS AND RECOMMENDATIONS 

An evaluation of this year's project was not possible 
because no data was received by the Office of Research Evaluation 
and Assessment. In the future, it is recommended that a greater 
effort is made to provide the data necessary for evaluation. 



*The effect size, developed by Jacob CDhen, is the ratio of the 
mean gain to the standard deviation of the gain. Thin ratio 
provides an index of improvement in standard deviation units 
irrespective of the tiize of the sample. According to Cohen, 0*2 
is a small E.S.; 0.5 is a moderate E.S., and 0.8 is considered to 
be a large E.S. Only effect sizes of 0.8 and above are considered 
to be educationally meaningful; reflecting the importance of the 
gains to the students' educational development. 
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#5001-48-93439 

RIVERDALE PREPARATORY ACADEMY; 1988-89 

School-Community Education Program 
Program Administrator: II. Morris Speiser 
Project Coordinator; Ralph Di Fiori 

Prepared by: 
Office of Research; Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Riverdale Preparatory Academy operates as a minischool 
for junior high school students in Community School District 
(C.S.D.) 10. At the minischool; students receive a full academic 
program that focuses on an intensive review of basic skills and 
special enrichment activities; and return to the main school only 
for subjects such as physical education and industrial artsv 

In 1988-89; the project operated at two sites ; Junior High 
Schools 45 and 141; and served some 270 seventh and ninth grade 
students who were selected by their schools on the basis of two 
criteria: evidence of average or above average ability and below 
grade level reading scores on citywide reading tests; and pup^ils 
who scored well on tests but achieved at a lower level than 
expected. Project activities included basic-skills instruction 
with an emphasis on the classics in literature, special writing 
projects; practical experiments with mathematics concepts, use of 
visual media; and attendance of diverse cultural events. 

The project had two objectives: student participants, who 
were reading on or below grade level, were expected to achieve a 
statistically significant mean gain cn annual citywide reading; 
and 75 percent of participants were expected to improve their 
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academic averages ; based upon major subject scores. 

School principals were responsible for the overall 
implementation of the program in their schools, and a teacher- 

in-charge was assigned to coordinate and supervise the mini- 
school activities. Funding of $22 thousand from the New York 
State Legislature was used for instructional supplies and 
enrichment activities. 
EVALUATION METHODOLOGY 

Evaluation activities focused on the analysis of student 
performance on the Degrees of Reading Power (D.R.P.) Test. To 
assess the first project objective, raw scores were converted to 
Normal Curve Equivalent (N.C.E.)* scores which express student 
performance relative to a national norm. Mean N.C.E. gains are 
interpreted as a measure of project impact on student 
achievement. Correlated t-tests were computed to establish if 
achievement differences were statistically significant. 



*N.C.E. scores are similar co percentile ranks, but unlike 
percentile ranks, are based on an equal-interval scale. Normal 
curve equivalent scores are based on a scale ranging from 1 to 99 
with a mean of 50 and a standard deviation of approximately 21. 
Because N.C.E. scores are equally spaced apart, arithm^atic and 
statistical calculations such as averaging are meaningful; in 
addition, comparisons of N*C*E* scores may be made across <4ifferent 
achievement tests. 
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Effect size (E.S.)*/ which indicates the educational 
meaningfulness of the mean gain or loss for ^ach comparison, was 
also calculated. Since two different criteria were used to 
select participants, pretest N.C.E. scores also served to 
distinguish between those students scoring at or below grade 
level (at or below 50 N.C.E.s) and those achieving above average 
reading scores (above 50 N.C.E.s) 

The second projecc objective was assessed through analysis 
of students' incoming final grades and their final grades at the 
end of the current year in the subject areas of english, social 
studies ; mathematics; and science. 
FINDINGS 

Complete test scores on the D.R.P. were reported for 197 
students, 58 percent of whom scored at or below grade level at 
pretest and the remaining 42 percent scored above grade level at 
pretest (see Table 1). By grade, 91 percent of the seventh grades 
students scored at or below grade level as compared to 37 percent 
of the ninth grade students who fell in this category. Overall, 
mean pretest score was 47.9 N.C.E.s and mean posttest score was 
53.9 N.C.E.s, for a mean gain of 6.0 N.C.E. points. This mean 

*The effect size, developed by Jacob Cohen, * the ratio of the 
mean gain to the standard deviation of th ^ain. This ratio 
provides an index of improvement in stana«rd deviation units 
irrespective of the size of the sample. iSccording to Cohen, 0.2 
is a small E.S., 0.5 is a moderate E.S., and 0.8 is considered to 
be a large E,S. Only effect sizes of 0.8 and above are 
considered to be educationally meaningful, reflecting the 
importance of the gains to the students' educational developments 
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Table 1 

Students' Mean N.C.E. Scores 







on 

Riverdale 


the D.R.P., by Group* 

Preparatory Academy, 


1988-89 












Pretest 


Post test 


Difference 




Group 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


E.S. 


At or 
Below 
Grade 
Level 


115 


36,7 


9.6 


44 .1 


10.0 


7.4" 


8.1 


0.9 


Above 
Grade 
Level 


82 


63.7 


7.9 


6/. 7 


9.1 


4.0" 


7.9 


0.5 


Total 


197 


47.9 


16.0 


53.9 


15. 1 


6.0" 


8.2 


0.7 



"students were broken into two groups: students at or below grade level at 
pretest (at or belov. 50 N.C.E.s); and students above grade 
level at pretest (above 50 N.C.E.s) . 

^hese mean gains were statistically significant at p<.05 

Overall; students achieved a laean gain of 6.0 N.C.E.s. This mean 
gain is statistically significant and represents a moderate effect 
size. 
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gain was statistically significant and represented a moderated 
effect size. Students scoring at or below grade level at pretest 
made the largest mean gain of 7.4 N.C.E.s. This gain was 
statistically significant and educationally meaningful. Students 
scoring above grade level at pretest also made a statistically 
significant mean gain, yet because of the large variation in 
scores, this represented only a moderate effect size. 

Students' mean grades for the major subject areas are 
presented in Tables 2 through 5. Overall mean gains and 
percentages of students making gains for each subject area are as 
follows: English mean gain was 3.9, with 58 percent of students 
ipaking a gain; social studies mean gain was 2.36, with 54.7 
percent of students achieving a gain; mathematics mean gain was 
1.4, with 45 percent of students achieving a gain; and science 
mean gain was 0.1, with 40 percent of students demonstrating a 
gain. Overall, students in grade nine achieved larger gains than 
students in grade seven across all four subject areas. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, The Riverdale preparatory Academy project was 
successful in meeting one of its objectives. With regard to the 
first objective, students achieved a statistically significant 
mean gain in reading, as measured by their performance on the 
annual citywide reading test. Students scoring at or below 
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Table 2 



Students' Mean Scores in 
English*, by Grade 
Riverdale Preparatory Academy, 1988-89 





Grade 


N 


Pretest Mean 


Posttest Mean 


Mean Gain 


Percent of Students 
Achieving Gain 


7 


98 


74.6 


75.2 


0.6 


41.8 


9 


127 


75.2 


81.6 


6.4 


70.9 


Total 




225 


74.9 


78.8 


3.9 


58.2 



Performance in the major subject areas was measured by a comparison of 
participants' incoming final grades with their final grades at the end of 
the current school year. 



Overall; 58.2 percent of all participating students achieved gains 
on their final grades in English. 
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Table 3 

Students' Mean Scores in 
Social Studies*, by Grade 
Riverdale Preparatory Academy, 1988-89 





Grade 


N 


Pretest Mean 


Posttest Mean 


Mean Gain 


Percent of Students 
Achieving Gain 


7 


98 


76.4 


74.6 


• -1.8 


40.8 


9 


127 


75.3 


80.9 


5.6 


65.4 


Total 


225 


75.8 


78.2 


2.4 


54.7 



^Performance in the major subject areas was measured by a comparison of 
participants' incoming final grades with their final grades at the end of 
the current school year. 

Overall; 54.7 percent of participating students achieved gains in 
their final social studies grades. 
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Table 4 

students' Mean Scores in 
Mathematics", by Grade 
Riverdale Preparatory Academy, 1988-89 



Percent of Students 
Grade N Pretest Mean Posttest Mean Mean Gain Achieving Gain 



98 



75.6 



74.6 



-1.0 



41.8 



127 



77.3 



80.9 



3.6 



47.2 



Total 225 



76.6 



78.2 



1.6 



44.9 



Performance in the major subject areas was measured by a comparison of 
participants' incoming final grades with their final grades at the end 
the current school year. 

Overall, 44.9 percent of participating students achieved gains on 
their final grades in mathematics. 
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Table 5 

Students* Mean Scores in 
Science^, by Grade 
Riverdale Preparatory Academy, 1988-89 



Percent of Students 
Grade N Pretest Mean Posttest Mean Mean Gain Achieving Gain 

7 98 75*3 75*7 ' 0*4 36.7 

9 127 75.2 75.1 -O.l 42.5 

Total 225 75.2 73.3 0.1 . 40.0 

''Performance in the major subject areas was measured by a comparison of 
participants' incoming final grades with their final grades at the end of 
the current school year. 

Overall; 40 percent of participating students achieved gains in 
their final science grades. 
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and emphasis of the program before expanding furt*ier. It would 
also be advisable to simplify and/or clarify selection criteria 
so that, in the future, the program can better concentrate its 
resources and improve its effectiveness. 
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